Rethinking Inductive Biases for Surface Normal Estimation